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Abstract 


Monotonicity properties of a general diagnostic model (GDM) are considered in this paper. Simple 
data summaries are identified to inform about the ordered categories of latent traits. The findings 
are very much in accordance with the statements made about the GPCM (Hernker, Sijtsma, 
Molenaar, & Junker, 1996, 1997). On the one hand, by fitting a GDM with equal slopes across 
items, the observed total score X + = ^ ■ =i x j demonstrates the monotone likelihood ratio 
(MLR) property in individual coordinate at the cost of losing model fit. On the other hand, 
fitting a GDM without slope restriction increases the model fit but by sacrificing MLR property. 
Trade-offs between these two situations should be considered in practice. 


Key words: General diagnostic model, monotone likelihood ratio, stochastic ordering of the 
manifest variables, stochastic ordering of the latent trait 
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1 Introduction 


Educational tests are often used to measure the position of students on a latent trait 9. 
Suppose that a test consists of J items each with m + 1 ordered categories. The score on item 
j is denoted as Xj and its realization is denoted as Xj(xj = 0,1,2,... , m). If rn = 1, items 
are dichotomous; if m > 1 the items are polytomous. Under the very mild conditions of latent 
trait unidimensionality (UD), local independence (LI) and item response functions that are 
nondecreasing functions of 6, the most widely used statistic for estimating 9 probably is the 
sum score, denoted as X + = Ylj=i Xj- Note that 0 < X + < mJ. Although there are some 
critiques concerning the use of A+ to estimate the latent trait (Samejima, 2001), A+ is still 
useful in communications between professionals and the general public, and it is also useful in 
the interpretation of item response models according to simple, monotone relationships between 
model components (van der Ark, 2005; Junker & Sijtsma, 2001). For example, Hernker, Sijtsma, 
Molenaar, and Junker (1996, 1997) considered in detail the monotone likelihood ratio (MLR) 
in the latent trait, stochastic ordering of the manifest sum score (SOM) by the latent trait and 
stochastic ordering of the latent trait(SOL) by the manifest sum score. For binary item responses, 
it was shown by Grayson (1988) and Huynh (1994) that under the conditions of UD, LI, and 
nondecreasing item response functions, the sum score X + demonstrates the MLR in 9. The MLR 
implies SOM and SOL (Lehmann, 1959, p. 74). For polytomous data, Hernker et al. (1997) 
demonstrated that the only two models that satisfy the MLR are the partial credit model (PCM; 
Masters, 1982) and the rating scale model (RSM; Andrich, 1978), that is, the generalization of 
the Rasch model (Rasch, 1960) to polytomous, ordinal data. SOM and SOL are two weaker 
nronotonicity conditions, and neither of them nor their combinations imply MLR (Junker, 1993). 
SOM is satisfied by all unidimensional polytomous models, and SOL usually does not hold for 
them in general. Little is known about the nronotonicity properties for multidimensional models 
for polytomous data. In this paper, all three properties are considered for a multidimensional 
polytomous model. Additionally, the nronotonicity condition proposed by Junker and Sijtsma 
(2001) is also considered. 

The organization of this paper is as follows: a general multidimensional polytomous model is 
introduced in Section 2. The nronotonicity properties are discussed for this model in Section 3. 
Finally, a brief discussion follows to direct subsequent data analysis activities. 
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2 General Diagnostic Model 

The general diagnostic model (GDM; von Davier, 2005; von Davier & Yamamoto, 2004) is a 
multidimensional extension of the PCM (Masters, 1982). The PCM has the following response 
function 


Pj{Xj = x\d) = 


exp [x9 + /?. 


xji 


1 + E™= l e Mv d + Pyj]' 

von Davier and Yamamoto decomposed the unidimensional latent trait into an item-dependent 
linear combination of K underlying traits a = (au, « 2 ,..., ax), 


exp [p xj + Yjk=i x ljkQjkOik\ 


P(Xj = x|a) = 


1 + E’yll exp \Pyj + Efc=1 yijkQjk^k] 

with dichotomous design matrix q^. j = 1 ,,J and k = 1 ,,K. This model makes it clear 
that P(Xj = x|c»!) is coordinate-wise monotone in a, if 7 ^ > 0. If each underlying trait only 
takes several discrete values specified in advance, the model becomes a cognitive diagnostic model 
that can be used to identify students’ skill mastery status. In some of the following sections, 
the continuous underlying traits are used to study the monotonicity properties. Because the 
cognitive diagnostic model is a special case for this general model, the statements from continuous 
underlying traits remain valid for the cognitive diagnostic model as well. 


3 Stochastic Ordering Properties 
3.1 Monotone Likelihood Ratio 

Let X + be the observed total score on the test. The likelihood ratio in the latent traits ot is 
defined as 


g(D,C;a ) = 


P(X + = D\ol) 


P{X + = C\ol) ’ 

where 0 < C < D < m J. The MLR of X + in ot holds for the GDM when the likelihood ratio is 
an increasing function of each coordinate aj~- Little is known about it. A weaker property related 
to the MLR is that this likelihood ratio is a nondecreasing function of individual latent trait a*,. 
Analogous to the derivations in Hernker et al. (1996), the MLR of X + in a holds if and only if 


Rd Rc J 

EE(Ei 


y =i v=i j =l 


/ 

TMa 0 


■jr. J IT TPOdTAH) — O’ 


where Rd and Rc are the numbers of score vectors that yield X + = D and X + = C respectively. 
Here x(/i) and x(y) denote realizations of vectors that yield X + = D and X + = C , respectively. 
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Denote 7 Tj x = P(Xj = x\ a), then KjxU) is the probability function of item j in vector x(g), and 
7 Tj c ( A( ) is the derivative of this function with respect to a*,. The same explanation applies to Ttj x {v) 
and its prime ^j x u,y Straightforward algebra shows that: 

din n jx _ 7 4 _ . . _ E£j= l exp[dyj + Efc=i y^jkqjkak}yijkqjk 

doik 1 + E^Li ex P[Ad + EjfeLi 

The summation of this ratio across all J items produces 


J 


E 



n jx 


yy x j r ijk ( ijk yy 

j=i i=i 


Eg=i exp[/3^- + ELi !rji.<r l k<u,jn.ik<i.ji- 
1 + E™=i ex P[Ad + Ef =1 VljkqjkUk ] 


Since the second term of the right side does not depend on data, we have 


B 


/ 


/ 

^jxii ') 


j 

ljkQjk{Xj(ix) ~ x j(v))- 

3 =1 


It is observed that only the items that require the attribute /c contribute to this summation. 
Because the derivative of g(D,C;a) with respect to a/., depends on D and C* only through the 
scores of those items who require the attribute k, the total score in the likelihood ratio should be 
the total score for those items (i.e. X + = Ylj-Q jk =i -E)- If 7 jk has the same value for all items 
(i.e. 7 jk = 7 fc > 0), then the MLR of X + in a*, holds. This can be easily seen that the equation 
above becomes t t 

V ' , ’ W) ^ m 1 = 71 y- (x _ XjM) > 0 


=1 EMm) ^jxiy) 


3 


3-<ljk = 1 


for all Rd * Rc possible combination of x(/i) and x{u). If 7 jk varies over items, the GDM does not 
in general imply the MLR of -X+ in a*,. In practice, the inferences towards each individual latent 
trait are often of interest, so we will focus on discussions of monotone properties in individual 
latent trait. Three weaker monotone properties are considered in the following. 


3.2 Stochastic Ordering of Manifest Variables 

Since the GDM satisfies local independence, monotonicity and low dimensionality, it follows 
immediately from Lemma 2 of Holland and Rosenbaum (1986) that SOM (Hernker et al., 1997) 
holds for the GDM. That is, P(X+ > x_|_|a) is nondecreasing in each coordinate o^. Note that 
X + is the total score of the test. Direct derivation makes this clear. Analogous to the derivation 
in Hernker et al. (1997), the derivative of P(X + > x+|a:) with respect to individual ak can 
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also be expressed as a sum of positive products where each product consists of one derivative 
P'(Xj > x\a) with respect to a k and J — 1 probabilities of the form 7 Ti x ,i / j: 

J 

P\Xj > x|a) \\nix. 

* 7 O' 

For the GDM model, the derivative P' (Xj > x\a) >0 needs to be shown. For notational 
convenience, let U x = exp \J3 X j + XX x ljkQjk<^k], and then the cumulative response function for 
item j in the GDM model can be expressed as 


P(Xj > x\a) 


U x + • • • + Um 
1 + U\ + U-2 + ... + U m 


X = 0, 1, ... , 777 ,. 


Direct algebra shows that 
3In P(Xj > x\a) P'(Xj > x|a:) 


da k 


P(Xj > x | q ;) 

u' x + u' x+ i + --- + u'm ET= 1 u' s 
Ux + U x +\ +... + u n 
r YT-X x u x+t t 

+X- 


"fjkQjki 


Ux +... + C/, 

xgr Ux+tt 

Ux + . . . + Um 

0 


__ r x ^2t=i- x U x +tt , ( 

7 jkQjki 1 | tt \ 


Er=i Us 
Eg* Uss 1 
i+Er=i^ J 

* - YT=i-x Ut+xt 

i+Er=i u s 1 

1 


y ] u x+t t\. 


1 + E s =i u s K u x + ... + u m i + Er=i u.'^ 

The first term in the bracket is positive, and the second term is nonnegative. So ^ Iu j > 0, 

and consequently > 0. Thus, SOM holds for the coordinate a k . In fact, this is 

true for every a k since the summation in the bracket is always positive. This is also true for a 
more restrictive sum score X + = Ej-Q.-j.x-- ^ is observed that item j will not be counted in 
this derivative if qj k = 0, thus indicating that only those items with qj k 0 are taken into the 
calculation of X+. 


3.3 Stochastic Ordering of Latent Variables 

Little is known about the SOL (Hernker, et. al, 1997) - P(a\ > ci,... ,cxk > ck\X + = x) 

when the latent trait is multidimensional. A weaker property related to SOL is that for any two 

values of a k , z\ > Z 2 , the ratio 

^ ^ _ j P(^k -2-1 1 data , ol r,..., cyfc—1, oi k j r \,..., otj ^) 

U(oc k 1 data , ou,..., 1, oi k j r \,..., oti \) 
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is nondecreasing in X + = ]Th . f;/=1 Xj, with all other parameters fixed. First, let ttq\ and 7 To 2 
be the prior distributions a k = z\ and a k = z 2 , respectively. The posterior odds of a k = z\ to 


atk = z 2 is 


J m 


, , Tn rr TT u , m 1 + £j,=i C l exp(mjk<ljkZ 2 ) \ 3 

r(zi,z 2 ) = - II II < exp [t'y jk q j k{zi ~ z 2 )\ y --r > 

^02 fJi [ 1 + E y =i C k expimjkQjkZi) J 

Here, C y = exp [f3 y j + yijlQjl a l\- For an item with q jk = 0, the score on that item does not 
count in the odds ratio since the ratio is 1 no matter what score it gets. Since the second part in 
the bracket does not depend on data, it will be cancelled out in the odds ratio. Let X(S) and 
X(T) be the response vectors to yield total score S and T with S > T. X(S)j and X(T)j are the 
responses to the item j in the vector X(S) and X(T), respectively. The odds ratio of these two 
response vector is 


IITT-t] = 6XP ^ ^ ^ M X ( S )j ~ 

' 15 2 ’ ' r<ijk=i 

If 7 jk = 7 fc, the odds ratio depends completely on the total score on those item who require the 
fcth attribute. That is, the posterior odds ratio for each coordinate a k is nondecreasing by this 
total score, and consequently, SOL holds. If 7 j k varies over items, SOL does not in general hold 
for this model. 


3.4 Junker and Sijtsman’s Property 

Finally, a new type of nronotonicity condition seems worthy of studying for the GDM. In a 
standard unidimensional IRT model, a higher level of the latent trait is associated with a higher 
probability of correct response. A corresponding property for the a cognitive diagnostic model 
could be the relationship between the number of task-relevant attributes the examinee masters 
and the probability of correct task performance (Junker & Sijtsma, 2001). This might require that 
the item response function in GDM model be nondecreasing in rriij = Ylk=i a kQjkJjk■ The odds 
for scores x\ > x 2 is calculated as 

h(xi,x 2 ) = = exp[P Xlj - p x2j + (xi - x 2 (^2'Y jk q jk a k ))]. 

i\-Aj — X2\OL) ^ 

Since the first term in the exponent will cancel out in the odds ratio of two different realizations 
of a, the odds ratio over different realizations of or. cc(l) and ct( 2 ), is 

KxuXiM 2)) = exp<11 - X2) L vUa(l) t - a(2)„), 
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where a(l) and a.(2) represent two vector of possible realization of a. If 7 = 77 the odds ratio 
is nondecreasing as the difference on the scores of task-relevant attributes increases. However, 
when 7 jk varies over the task-relevant attributes, this nronotonicity does not hold in general. 

4 Discussion 

Relating the latent traits to simple and useful data summaries is important when 
communicating to the public. For example, SOL (Hernker et al., 1997) was considered, which 
asserted that the higher score on the items that require latent attribute k, the easier it is to get 
a higher category in this attribute. For inference on the latent traits, SOL by the sum score is 
more useful than SOM of the sum score. The MLR is more stringent than these two. When 
Ijk = 7 fc, the MLR holds and this implies that both SOM and SOL hold for GDM model. In 
general, MLR does not hold for GDM because 7 ^ varies over items. The same is true for the SOL 
property. These statements are very much in accordance with those of the GPCM (Hernker et 
al., 1996, 1997; see Table 1). Under the GDM, similar to the GPCM, some weighted sum score 
^Zj- qjk= \ljk x ji n °t Yhj-q jk =i x j-> h as the MLR property. By fitting a stringent GDM with equal 
slope across items, the observed total score X + = =1 Xj has the MLR property. However, the 

model fit might be lost. By fitting the GDM without the slope restriction,the model fit increases, 
but the MLR property is sacrificed. In practice, a trade-off is necessary between these two choices. 

Table 1 


Monotonicity Properties of Models 


Properties 

Rasch, PCM, RSM, 

2PL/3PL, GPCM 

GDM with equal slope GDM 

SOM 

X 

X 

X X 

SOL 

X 


X 

MLR 

X 


X 


As mentioned above, SOL generally does not hold for the GDM. However, it is not clear how 
seriously the violation of SOL will affect the inference of the latent traits in a fitted model. It is 
incumbent on GDM model users to check that SOL holds in the fitted model before asserting that 
a higher sum score corresponds to a higher category in the latent traits. A step in future research 
might be a thorough simulation study to show the severeness of this violation. 
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